Constraints on Tree Structure in Concept Formation

We describe ARACHNE, a concept formation system that, uses explicit constraints on tree structure and local restructuring operators to produce well-formed probabilistic concept trees. We also present a quantitative measure of tree quality and compare the system's performance in artificial and natural domains to that of COBWEB, a well-known concept formation algorithm. The results suggest that ARACHNE frequently constructs higher-quality trees than COBWEB, while still retaining the ability to make accurate predictions. 1 Background and Motivation The task of concept formation involves the incremental acquisition of concepts from unlabeled training instances (Fisher &; Pazzani, in press). Much of the recent research on this topic builds on Fisher's (1987) COBWEB. Fisher's system assumes that each instance is described as a conjunction of attribute-value pairs, and employs a probabilistic representation for concepts. In particular, COBWEB represents each concept Ck as a set of attributes Ai and a subset of their possible values V ij. Associated with each value is the conditional probability of that value given membership in the concept, P(A i = V ij \C k). In addition, each concept has an associated probability of occurrence, P(C k)-COBWEB organizes its conceptual knowledge into a hierarchy, with nodes partially ordered according to their generality; thus, the root node summarizes all instances that have been observed, terminal nodes correspond to single instances, and intermediate nodes summarize clusters of observations. COBWEB integrates the processes of classifying instances and incorporating them into memory. The system sorts each new instance I down the hierarchy, starting at the root, locating nodes that summarize classes into which the instance fits well. At a given node TV, COBWEB retrieves all children and considers placing the instance in each child node C in turn; it also considers creating a new child based on the instance. The algorithm uses an evaluation function, category utility (Gluck & Corter, 1985), to determine the "best" resulting partition , then incorporates the instance into memory accordingly. The system then recurses, sorting the instance through memory until it produces a disjunct or reaches a terminal node. Other research on concept formation *Also affiliated with Sterling Federal Systems. Gennari &: Allen, in press)-an integrated cognitive architecture use COBWEB as the underlying engine for classification and concept formation. ICARUS invokes Fisher's algorithm to acquire primitive concepts, which in turn serve as background knowledge for the rest of the system. In theory, COBWEB produces structures containing concepts that correspond …